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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments filed 14 June 2004 have been fully considered but they are 
not persuasive. 

2. The applicant argues that there is no motivation to combine Kaiser and 
Blackketter to teach, "a plurality of annotations having equal timing information," as 
recited in claim 28. The examiner disagrees. In response to applicant's argument that 
there is no suggestion to combine the references, the examiner recognizes that 
obviousness can only be established by combining or modifying the teachings of the 
prior art to produce the claimed invention where there is some teaching, suggestion, or 
motivation to do so found either in the references themselves or in the knowledge 
generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071 , 5 
USPQ2d 1596 (Fed. Cir. 1988) and In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. 
Cir. 1992). In this case, the motivation to combine is derived from the knowledge 
generally available to one of ordinary skill in the art. In other words, one of ordinary skill 
in the art would readily recognize that life spans of triggers being of equal timing would 
be advantageous in that the users of the system would be able to easily identify the 
triggers and respond accordingly. 

The applicant further disagrees "that one would readily recognize that equal 
timing is an inherent feature of the system." The examiner disagrees; Blackketter 
teaches different methods of having life spans for triggers, wherein the timing can be 
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either equal or unequal timings, and the examples as disclosed show the same equal 
timing but with different time bases (col. 9, II. 48-67), which equates to equal timing. 

The applicant previously cancelled claims 1-11 as claims 64-74, respectively, but 
has provided no specific argument as to their reasons for placing the application in 
condition for allowance. 

3. Applicant's arguments with respect to claims 52-62 have been considered but are 
moot in view of the new ground(s) of rejection. 

Further, the applicant has made no specific arguments to newly introduced 
claims 52-62. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 26-28, 33, 35-43. and 49-51 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 6,615,408 to Kaiser et al. in view of U.S. Patent 
6,415,438 to Blackketter et al. 

Regarding claim 26, Kaiser is silent on the timing information comprising one of a 
timestamp, timecode, frame numbering, or global time of day. Blackketter teaches 
inserting triggers with a time attribute (col. 4, II. 64-67), such as a frame number (col. 6, 
II. 1 6-22). Therefore, it would have been obvious to one of ordinary skill in the art at the 
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time the invention was made to modify Kaiser by using time information such as a frame 
number as taught by Blackketter in order to eliminate a delay loop (Blackketter: col. 2, II. 
59-61). 

Regarding claim 28, Kaiser teaches a reproduction apparatus (fig. 1, label 1300) 
in communication with a broadcast channel (col. 4, II. 59-67; col. 5, II. 22-31), a display 
(fig. 1 , label 1200) (claimed display device). Kaiser teaches a receiver decoding a 
digital signal to recover a video signal (such as high definition television formats, see 
col. 5, II. 30) and annotation data (fig. 2). Kaiser teaches displaying annotation 
information in response to a viewer request (fig. 5, col. 9, II. 37-65). which is available 
on a frame-by-frame basis. 

Kaiser teaches is silent on the annotation data having equal timing information. 
Blackketter teaches life spans for triggers, which clearly permit the equal timing 
information of annotation data (col. 10, II. 1-12), in that one would readily recognize that 
equal timing is an inherent feature of the system. Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
Kaiser by having equal timing information for annotation data as taught by Blackketter in 
order to present the triggers to the user in an effective and predictable fashion thereby 
increasing user interactivity. Kaiser teaches the user selecting which of the annotations 
is to be displayed (col. 9, II. 37-65). 

Regarding claim 27, Kaiser clearly synchronizes the placement zones and the 
video in response to timing information in order to place the placement zones over the 
proper location, such as a car as shown in figures 6A-6D. 
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Regarding claim 33, Kaiser teaches product purchase actions (col. 12, II. 29-54), 
which reads on infomnation regarding goods and services. 

Regarding claim 35 and 36, Kaiser teaches a placement zone which references a 
product being displayed, wherein the placement zone is a location of an object 
(Abstract). 

Regarding claims 37-39, Kaiser teaches placement zones for various frames (fig. 
2, col. 6, II. 9-17), wherein the placement can track an image at the upper left corner 
(col. 10, II. 34-38). Whereas Kaiser is silent on a location reference at the centroid pixel, 
Official Notice is taken that a center position is well known in the art, such as tracking an 
object by the center position. Therefore, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify Kaiser by using a center 
position (as a reference point) in order to properly track an image thereby increasing the 
effectiveness of the placement zone. 

Regarding claims 40-42, Kaiser teaches location and shape information such as 
the shape and location of the car (col. 10, II. 34-38), see figures 6A-6D, wherein the 
visual highlight (fig. 6B, label 6500) is a graphical overlay and has an outline of the car. 

Regarding claim 43, Kaiser is silent on a mathematical representation of set of 
pixels. Official Notice is taken that mathematical representation of pixels are well known 
in the art, such as vectors used in video encoding in order to reduce the bandwidth by 
reducing duplicity of the pixels. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by using a 
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mathematical representation in order to save bandwidth and process the infomnation at 
the client side. 

Regarding claim 49, Kaiser teaches a back channel (fig. 1, label 1400-data 
network). 

Regarding claim 50, Kaiser teaches product selection or information (col. 3, II. 3- 
5), collecting user infomnation (col. 14, II. 3-21), and accepting infonnation regarding a 
commercial transaction and completing the transaction (fig. 9). 

Regarding claim 51 , Kaiser performs a 'secure transaction verification' procedure 
(fig. 9, label 9300), which in the broadest reasonable sense has some information 
regarding a viewer identifier in order to identify the viewer. 

Regarding claim 52, Kaiser teaches plural annotations, which as discussed in 
claim 28 have equal timing. Further, Kaiser teaches that the plurality of annotations 
would be different in that the system will need to differentiate among plural annotations 
in that they will have different masks and provide different information. 

6. Claims 53-56 and 58-63 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 6,615,408 to Kaiser et al. in view of U.S. Patent 
6,282,713 to Kitsukawa et al. 

Regarding claim 53, Kaiser teaches a reproducing apparatus (fig. 1, label 1300), 
which reads on a receiver that receives video (col. 5, II. 10-31) and action resource data 
(claimed annotation data) (col. 5-6, II. 55-8), wherein the action resource data includes 
overlay information as shown in figure 6B (col. 10, II. 9-41), selectable actions (claimed 
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object data appearing on a portion of the video) (fig. 6A-6D), and timing information (fig. 
2, col. 6, II. 9-64). Whereas Kaiser does not explicitly teach a data store storing at least 
a portion of the received annotation data, the system inherently stores a portion of the 
data in buffers, memory, or processor registers in order to process the received 
information. Kitsukawa teaches synchronizing the display of the overlay and object data 
with portions of the video, but is silent on synchronizing the image data responsive to a 
user command. 

In analogous art, Kitsukawa teaches an integrated receiver/decoder (IRD), which 
equates to a viewer interaction device (fig. 2 and 3, label 2; col. 5, II. 1-28), wherein 
Kitsukawa teaches the user selecting an advertisement mode, wherein the annotation 
data is displayed to the viewer (col. 6-7,11. 65-21), which equates to displaying 
annotation data on said display device in response to a viewer activating said viewer 
activating said viewer interaction device (fig. 5-11). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Kaiser by performing the display of the 
annotation data in response to a user command as taught by Kitsukawa in order to 
enable the user to further diversify the system by customizing the system to the specific 
user's desires. 

Regarding claim 54, as shown in figure 6B, Kaiser visually identifies the object. 
Regarding claim 55, as shown in figure 6B, Kaiser teaches the location of the 

object. 

Regarding claim 56, as shown in figure 6B, Kaiser teaches shape information. 
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Regarding claim 58, Kaiser teaches in-band techniques for receiving annotation 
data (col. 5, II. 45-64. col. 6-7, II. 65-4). 

Regarding claim 59, Kaiser teaches annotation data over a broadcast 
transmission with the video stream (col. 5, II. 45-54). 

Regarding claim 60, Kaiser teaches sending the annotation data over a 
broadcast transmission with the video stream (col. 5, II. 45-54), which clearly would be 
received prior to the video stream in order to synchronize the display of the annotation 
data. 

Regarding claim 61 , Kaiser teaches sending the data over a separate 
transmission medium from the video (col. 5, II. 45-54), which equates to an out of band 
signal. 

Regarding claim 62, Kaiser teaches sending the data over a data communication 
network (col. 5, II. 45-54). 

Regarding claim 63, Kaiser teaches permitting the user to engage in purchasing 
an item (col. 12, II. 29-65). 

7. Claim 57 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 6,615,408 to Kaiser et al. and U.S. Patent 6,282,713 to Kitsukawa et al. in view 
of U.S. Patent 6,41 5,438 to Blackketter et al. 

Regarding claim 57, Kaiser teaches displaying the annotations, but both Kaiser 
and Kitsukawa are silent on second timing information for removing the annotation data 
from the data store. Blackketter teaches expiring triggers (col. 3, II. 13-22, col. 10, II. 24- 
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35). Therefore, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify Kaiser and Kitsukawa by indicating an expiration time 
in order to ignore invalid triggers (Blackketter: col. 10, II. 24-35). 

8. Claim 34 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 6,615,408 to Kaiser et al. and U.S. Patent 6,415,438 to Blackketter et al. in view 
of U.S. Patent to Hidary et al. 

Regarding claim 34, Kaiser is silent on non-commercial information, which is 
taught by Hidary (col. 2, II. 43-48, col. 8, II. 18-44). Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
Kaiser by providing non-commercial information as taught by Hidary in order to provide 
additional information to the user, such as web sites, thereby increasing user- 
interactivity. 

9. Claim 64 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 5,774,664 to Hidary et al. 

Regarding claim 64, Hidary teaches a content creation device (fig. 1 , label 4) 
which clearly has a video source and produces a transport stream, which is inherent to 
MPEG-2 (col. 4, II. 36-39). After the video program is created, Hidary teaches an 
annotation source, such as Uniform Resource Locators (URLs) (col. 4, II. 40-47). 
Whereas Hidary teaches inserting the URL into the vertical blanking Interval (VBI) of a 
signal, Hidary clearly teaches MPEG-2 transmissions, which do not have a VBI. Hidary 
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does not explicitly teach a multiplexer. Official Notice is taken that multiplexers are well 
known in the art, such as multiplexing plural video channels onto a single physical 
channel. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Hidary by using a multiplexer in order to transmit 
the URL as a data packet within the transport stream thereby efficiently sending 
information within a given channel. The embedded URL contains timing information in 
order to present the URLs at the appropriate time (col. 5, II. 21 - 33, col. 7, II. 31-33), 
which reads on synchronizing the annotation data with the video signal. 

10. Claim 65 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 5,774,664 to Hidary et al. in view of U.S. Patent 6,415,438 to Blackketter et al. 

Regarding claim 65, Hidary teaches inserting the URL at the time it is needed; 
but is silent on the timing information comprising one of a timestamp, timecode, frame 
numbering, or global time of day. Blackketter teaches inserting triggers with a time 
attribute (col. 4, II. 64-67), such as a frame number (col. 6, II. 16-22). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Hidary by using time information such as a frame number as taught by 
Blackketter in order to eliminate a delay loop (Blackketter: col. 2, II. 59-61). 

1 1 . Claims 66-74 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
U.S. Patent 5,774,664 to Hidary et al. in view of U.S. Patent 6,615.408 to Kaiser et al. 
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Regarding claims 66 and 67, Hidary teaches a URL which reads on textual data; 
but Is silent on mask data and location information. Kaiser teaches a placement zone 
which references a product being displayed, wherein the placement zone is a location of 
an object (Abstract). Therefore, it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Hidary by using a placement zone 
(claimed mask data) including location information as taught by Kaiser in order to 
increase interactivity and enable the user to gather more infomnation on a product. 

Regarding claims 68-70, the combination of Hidary and Kaiser has been 
explained above. Further, Kaiser teaches placement zones for various frames (fig. 2, 
col. 6, II. 9-17), wherein the placement can track an image at the upper left corner (col. 
10, II. 34-38). Whereas Hidary and Kaiser are silent on a location reference at the 
centroid pixel. Official Notice is taken that a center position is well known in the art, such 
as tracking an object by the center position. Therefore, it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to modify Hidary and 
Kaiser by using a center position (as a reference point) in order to properly track an 
image thereby increasing the effectiveness of the placement zone. 

Regarding claims 71-73, the combination of Hidary and Kaiser has been 
explained above. Kaiser teaches location and shape information such as the shape and 
location of the car (col. 10, II. 34-38), see figures 6A-6D. wherein the visual highlight (fig. 
6B, label 6500) is a graphical overlay and has an outline of the car. 

Regarding claim 74, Hidary and Kaiser are silent on a mathematical 
representation of set of pixels. Official Notice is taken that mathematical representation 
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of pixels are well known in the art, such as vectors used in video encoding in order to 
reduce the bandwidth by reducing duplicity of the pixels. Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
Hidary and Kaiser by using a mathematical representation in order to save bandwidth 
and process the infomiation at the client side. 

Conclusion 

12. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Andrew Y Koenig whose telephone number is (703) 
306-0399. The examiner can normally be reached on M-Th (7:30 - 6:30). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Christopher Grant can be reached on (703) 305-4755. The fax phone 
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number for the organization where this application or proceeding is assigned is 703- 
872-9306. 

Infomation regarding the status of an application may be obtained from the 
Patent Application Infomriation Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status infomriation for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 
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